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Abstract 

We study the problem of the reconstruction of a Gaussian field defined in [0, 1] using A'^ sensors de- 
ployed at regular intervals. The goal is to quantify the total data rate required for the reconstruction of the 
field with a given mean square distortion. We consider a class of two-stage mechanisms which a) send 
information to allow the reconstruction of the sensor's samples within sufficient accuracy, and then b) use 
these reconstructions to estimate the entire field. To implement the first stage, the heavy correlation between 
the sensor samples suggests the use of distributed coding schemes to reduce the total rate. We demonstrate 
the existence of a distributed block coding scheme that achieves, for a given fidelity criterion for the re- 
construction of the field, a total information rate that is bounded by a constant, independent of the number 
N of sensors. The constant in general depends on the autocorrelation function of the field and the desired 
distortion criterion for the sensor samples. We then describe a scheme which can be implemented using 
only scalar quantizers at the sensors, without any use of distributed source coding, and which also achieves 
a total information rate that is a constant, independent of the number of sensors. While this scheme oper- 
ates at a rate that is greater than the rate achievable through distributed coding and entails greater delay in 
reconstruction, its simplicity makes it attractive for implementation in sensor networks. 

1 Introduction 

In this paper, we consider a sensor network deployed for the purpose of sampling and reconstructing a spa- 
tially varying random process. For the sake of concreteness, let us assume that the area of interest is repre- 
sented by the line segment [0, 1], and that the for each s e [0, 1], the value of the random process is X{s). 
For example, X{s) may denote the value of some environmental variable, such as temperature, at point s. 
A sensor network, for the purpose of this paper, is a system of sensing devices (sensors) capable of 

1 . taking measurements from the environment that they are deployed in, and 

2. communicating the sensed data to a fusion center for processing. 

The task of the fusion center is to obtain a reconstruction {X{s), s E [0, 1]} of the spatially varying process, 
while meeting some distortion criteria. 

There has been great interest recently in performing such sensing tasks with small, low power sensing 
devices, deployed in large numbers in the region of interest HI, ||2l, IS lU. This interest is motivated by the 
commercial availability of increasingly small and low-cost sensors which have a wide array of sensing and 
communication functions built in (see, for example, (Si), and yet must operate with small, difficult to replace 
batteries. 

Compression of the sensed data is of vital importance in a sensor network. Sensors in a wireless sensor 
network operate under severe power constraints, and communication is a power intensive operation. The 
rate at which sensors must transmit data to the fusion center in order to enable a satisfactory reconstruction 
is therefore a key quantity of interest. Further, in any communication scheme in which there is an upper 
bound (independent of the number of sensors) on the amount of data that the fusion center can receive per 
unit time, there is another obvious reason why the compressibility of sensor data is important - the average 
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rate that can be guaranteed between any sensor and the fusion center varies inversely with the number of 
sensors. Therefore, any scheme in which the per-sensor rate decreases slower than inversely with the number 
of sensors will build backlogs of data at sensors for large enough number of sensors. 

Environmental variables typically vary slowly as a function of space and it is reasonable to assume that 
samples at locations close to each other will be highly correlated. The theory of distributed source coding 
(B, Q, ID) shows that if the sensors have knowledge of this correlation, then it is possible to reduce the 
data-rate at which the sensors need to communicate, while still maintaining the property that the information 
conveyed by each sensor depends only on that sensor's measurements. Research on practical techniques 
(||9l, ifTOl , ifTTl . lfT2l . ifTSi ) for implementing distributed source coding typically focuses on two correlated 
sources, with good solutions for the many sources problem still to be developed. Thus, in our work, we 
attack the problem at hand using the available theoretical tools which have their origins in ||6l. 

This approach has been taken earlier in HI and ||2], which investigate whether it is possible to use such 
distributed coding schemes to reduce the per-sensor data rate by deploying a large number of sensors at 
closely spaced locations in the area of interest. In particular, it is investigated whether it is possible to 
construct coding schemes in which the per-sensor rate decreases inversely with the number of sensors. The 
conclusion of |[T], however, is that if the sensors quantize the samples using scalar quantizers, and then 
encode them, the sum of the data rates of all sensors increases as the number of sensors increases (even with 
distributed coding), and therefore the per-sensor rate cannot be traded off with the number of sensors in the 
manner described above. 

Later, though, it was demonstrated in |fT4l that there exists a distributed coding scheme which achieves 
a sum rate that is a constant independent of the number of sensors used (so long as there is a large enough 
number of sensors). The per-sensor rate of such a scheme therefore decreases inversely with the number of 
sensors, which is the trade-off of sensor number with per-sensor rate that was desired, but shown unachievable 
with scalar quantization, in (|T|. Results similar to those of lfT4l for the case when a field of infinite size 
is sampled densely have since appeared in |3J. However, a question that still appears to be unresolved is 
whether it is possible to achieve a per-sensor rate that varies inversely with the number of sensors using a 
simple sensing (sampling, coding, and reconstruction) scheme. 

This paper is an expanded version of 1141 . We describe the distributed coding scheme of 1141 in detail, and 
then study another sampling and coding scheme which achieves the desired decrease of per-sensor rate with 
the number of sensors. The two main properties of this scheme are that (1) it does not make use of distributed 
coding and therefore does not require the sensors to have any knowledge of the correlation structure of the 
spatial variable of interest, and (2) it can in fact be implemented using only scalar quantizers at the sensors 
for the purpose of coding the samples. The scheme utilizes the fact that the sensors are synchronized, which 
is already assumed in the models of HI, |l2|, |I3], and is easily achievable in practice. Since scalar quantizers 
are easily implementable in sensors with very low complexity, this paper shows that it is possible achieve 
per-sensor rates that decrease inversely with the number of sensors with simple, practical schemes. 

A brief outline of this paper is as follows: We pose the problem formally and establish notation in Sec- 
tion 11.11 We study the achievability of the above tradeoff with a distributed coding scheme in Section |2] 
and compare the rate of this coding scheme with that of a reference centralized coding scheme in Section |3] 
We describe the simple coding scheme mentioned above in Section |4] Some numerical results are presented 
in Section|5] We make some concluding remarks in Section|6] 

1.1 Problem statement 

1.1.1 Model for the spatial process 

We take a discrete time model, and assume that the spatial process of interest is modeled by a (spatially) 
stationary, real-valued Gaussian random process, X^^\s) at each time i, where s is the space variable. The 
focus of this paper is the sampling and reconstruction of a finite section of the process, which we assume 
without loss of generality to be the interval [0, 1]. We follow conventional usage in referring to the spatial 
process X^*) = {X^'^{s),s G [0, 1]} as the, field at time i. 

We assume that the field X^*) at time i is independent of the field X'^-'^ for any j ^ i, and has identical 
statistics at all times. (In what follows, we omit the time index when we can do so without any ambiguity.) 
For simplicity, we assume that X is centered, £[X(s)] = 0, and that the variance of X{s) is unity, for all 
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s G [0, 1]. The autocorrelation function of the field is denoted as 



p{t)^8[X{s)X{s + t)]. 



Following common usage, we sometimes refer to p as the correlation structure of the field. Clearly, p(0) = 1, 
and p{t) < 1 for any r. We need only mild assumptions on the field X: 

1 . We assume that X is mean-square continuous, which is equivalent to the continuity of p at (see, for 
example, jTSl ). 

2. We assume that there is a neighborhood of in which p is non-increasing. 

Note that all results in this paper extend to fields in higher dimensions. We restrict the exposition to 
one-dimensional fields for clarity and to avoid the tedious notation required for higher dimensional fields. 

1.1.2 Assumptions on the sensor network 

We assume that N sensors are placed at regular intervals in the segment [0, 1], with sensor k being placed at 
Sk ~ '^2N^ ^'^^ ^ ~ 1, 2, . . . , iV. Sensors are assumed to be synchronized, and at each time i, sensor k can 
observe the value X*^*^ (sfe) of the field at its location, for each k. Sensor k encodes a block of m observations, 
[X(i)(sfe),^^^Hs/c). • ■ ■iX'^"^'>{sk)] into an index 4 chosen from the set {1, 2, . . . , [e'"^'=J }, where Rk is 
the rate of sensor k, which we state in the units of nats per discrete time unit. We assume that the blocklength 
m is the same at all sensors. The messages of the sensors are assumed to be communicated to the fusion 
center over a shared, rate constrained, noiseless channel. The fusion center then uses the received data to 
produce a reconstruction X^*) (s) of the field. 

A coding scheme is a specification of the sampling and encoding method used at all sensors, as well as 
the reconstruction method used at the fusion center 

1.1.3 Error criterion 

We refer to £{X^'''> (s) — X^*' (s))^ as the mean square error (MSB) of the reconstruction of the field at point 
s and time i. We measure the error in the reconstruction as the average (over a blocklength) integrated MSB, 
which is defined as 



We study coding schemes in which, for all large enough blocklengths m and a specified positive constant 
Dnet , the fusion center is able reconstruct the field with an integrated MSB of less than Dnet , that is, schemes 
for which 



1.1.4 Sum rate 

In this paper, we describe coding schemes in which for any given value of Dnet in ©, the sum rate, X^feLi ^k, 
is bounded above by some constant R independent of the number N of sensors. The bound R may in general 
depend on Dnet- This allows the per-sensor rate can be traded off with the number of sensors, so that for all 
N large enough, the rate of each sensor is no more than a constant multiple of . 

1.2 Contributions 

Our main contributions are: 

1 . We prove the existence of a distributed coding scheme in which, under the assumption that the correla- 
tion structure is known at each sensor, a sum rate that is independent of the number of sensors N can 
be achieved. 




(1) 



lim JMSE{m) < D, 



net • 



(2) 
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2. We design a simple coding scheme which can be implemented using scalar quantization at sensors, 
which does not require the sensors to have any information about the correlation structure, and which 
makes use of the fact that the sensors are synchronized to achieve a sum rate that is a constant indepen- 
dent of N. 

The latter scheme has the advantage of being simple enough to be implementable even with extremely 
resource-constrained sensors. However, the sum-rate achievable through this scheme is in general greater 
than the sum-rate achievable through distributed coding. Also, unlike distributed coding, this scheme entails 
a delay that increases with the number of sensors in the network. 



2 Distributed coding 

In this section we describe a distributed coding scheme which achieves the desired scahng. 

2.1 Encoding and decoding 

The scheme consists of N encoders, where fk is the encoder at sensor k, and N decoders, {gk}k=i 

at the fusion center For each k, the rate of fk is assumed to be Rk, and fk maps the block 

[X^'\sk),X^'\sk),...,X^"^\sk)] 

of samples to an index Ik chosen from {1,2,..., [6™^*=] }, which is then communicated to the fusion center 
While the output of encoder k may not depend on the realizations of the observations at any other sensor 
i ^ fc, it is assumed that all sensors have knowledge of the statistics of the field (in particular, the function p 
is assumed known at each sensoiQ) and utilize this information to compress their samples. The decoders may 
use the messages received from all encoders to produce their reconstruction: 

X^'^- '"'(sfc) = 9k{fi{X^'-- • • • , fN(X^''- '"'(siv))), 

where X'-^'- •'"^(sfe) is shorthand for [X'-^^Sk), X'-^^Sk), . . . , ^('"'(sfe)], for = 1, . . . , TV and similarly 
fovX. 



2.2 Reconstructing the continuous field 

The reconstruction of the field for those values of s e [0,1] where there are no sensors is done in a two-step 
fashion as follows. In the first step, the estimates X{sk) of sensor samples are obtained as described above. 
Then, the value of the field between sensor locations is found by interpolation. 

The interpolation X{s) for s ^ {sk\k = 1, . . . , N} is based on the minimum MSB estimator for X{s) 
given the value of the sample closest to s. Formally, for any s, define n(s) = if s S [j^, as the 
location of the sample closest to s. Then, given X{n{s)), the minimum MSB estimate for X{s) is given by 
£[X {s)\X {n{s))] = p{s — n{s))X{n{s)). The reconstruction of the field at the fusion center is obtained by 
replacing X{n{s)) in this estimate with the quantized version X{n{s)), 

Xis)=p{s~n{s))Xin{s)). (3) 

While this two-step reconstruction procedure is not optimal in general, it suffices for our purposes. 



2.3 Error analysis 

Define 

fe=l i=l 



N 



'in practice, the sensors need only know the vector |^p(-^),p(-^),...,p (^Tv^)] ■ 
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Using the upper bound found in equation (|2TI ) (Appendix |A]l on the error of the coding scheme described 
above, we see that lim,„ Jmse{iti) < Dnet is met if lim„i J^ssi''^) — D'{N), where 

given that N is large enough so that 1 ^ ( 5^) ^ D„et- It is easy to see that D'{N) approaches Dnet from 
below as iV ^ cx). 



2.4 Sum rate 

We now study the sum rate of the distributed coding scheme discussed above. We begin with finding the 
encoding rates required for achieving 

Yim. JMSEifn) <D, (6) 

m 

for some constant D. 

The rate region Tl{D) is defined as the set of all A^— tuples of rates (i?i, i?2, • ■ • , Rn) for which there 
exist encoders fk and decoders gk, for k ~ 1, . . . ,N, such that ^ can be met. If a rate vector belongs to the 
rate region, we say that the corresponding set of rates is achievable. 

The rate-distortion problem in (|6]l is a Gaussian version of the Slepian-Wolf distributed coding prob- 
lem 161. Until recently, the rate region for this problem was not known for even 2 sources. An achievable 
region for two discrete sources first appeared in |,16| , and was extended to continuous sources in 17|. The 
extension to a general number of Gaussian sources appears in ifTTl . The two-source Gaussian distributed 
source coding problem was recently solved in f8|, where the achievable region of lfT6l was found to be tight. 
The rate region is still not known for more than 2 sources. We use the achievable region found in 117] . 

Though the result is stated in ifTTl for individual distortion constraints on the sources, the extension 
to a more general distortion constraint is straightforward. We state the achievable region for distributed 
source coding in the form most useful to us in Theorem [T] below. In the statement of the theorem, we 
use A B C to denote a Markov-chain relationship between random variables A, B and C, that is, 
conditioned on B, A is independent of C. Also, for any S C {!,..., N}, X5 denotes the vector of those 
sources the indexes of which lie in the set S and S'^ denotes the complement of the set S. 

Theorem 1 Tl{D) D Tlin{D), where TZin{D) is the set of N —tuples of rates for which there exists a vector 
U e of random variables that satisfies the following conditions. 

1. V C {1, 2, . . . , N}, U5 ^ Xs ^ Xsc ^ Use 

2. V5C{l,2,...,Af}, E^^^i?, >/(Xs;Us|Usc). 



3. 3 X(U) such that 



1 ^ r 

(x(,sO-X(.,)(U) 

i=l L 



< D. (7) 



Note that each of the rate-constraints in Theorem[T]forms some part of the boundary of the achievable region 
TZin (see, for example, IfTTl ). In particular, the constraint on the sum rate is not implied by any other set of 
constraints. 

Constructing a vector U satisfying the conditions of Theorem[T]corresponds to the usual construction of 
a forward channel for proving achievability in a rate-distortion problem. For each i, Ui can be thought of as 
the encoding of X{si). 

We now construct a U that would suffice for our purposes. Consider a random vector Z G that is 
independent of X, and has a Gaussian distribution with mean and covariance matrix pi, where / is the 
identity matrix. Then U = X + Z satisfies the Markov chain constraints of Theorem[T] To find a good bound 
on the sum rate, we now find a lower bound on the variance p for which there exists an estimator X(X + Z) 
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which satisfies condition (|7|. Since X + Z is jointly Gaussian with X, the estimator which minimizes the 
MSE in ^ is the linear estimator, 

X(X + Z) = Sx(x+z)Sx+z(X + Z), (8) 

where Sx(x+z) = f [X(X + Z)^] and Sx = £[XX^]. Let Pmax(^, D, p) be the largest value of p for 
which the MSE achieved by this estimator satisfies Q. We prove below that for large enough N, Pmax grows 
faster than linearly with N . 

Lemma 1 Let p{t) be a symmetric autocorrelation function such that lim^^o p{t) = 1 f^nd a threshold 
9 > exists for which 

1- 1 > p{t) > p{0) >OifT e (0, 9) and 

2. the inequality 1 — p'^{9)/{l + 9) < D holds. 

Then 

liminf ^pn,UN,D,p) > 9^. 

Note: The second condition can be met for all D > since 1 — (9) / {1 + 9) ^ as 9 ^ 0. 
Proof: We call a value of p allowable if the expected reconstruction error in dTji, with U = X + Z, is less 
than D. We find the largest p for the error criterion: £[{X{si) — X{si))'^] < D for each i E {1, . . . , N}, 
which is more stringent than the average error requirement of 

Let us consider the estimation of X{si). Since X{si) is the best linear estimate of X{si) from the 
data X + Z, any other linear estimator cannot result in a smaller expected MSE. We take advantage of this 
observation and choose a linear estimator that although suboptimal, is simple to analyze and yet suffices to 
establish the lemma. 

Our estimator for X(si) shall be the scaled average a J^kknb X{si) + Zi, where a is a parameter to be 
optimized shortly. To estimate X{si) for i 0, simply substitute the samples used with those whose indexes 
lie in the set {i + 1, • • • ,i + N9} (or, for samples at the right edge of the interval [0, 1], {i — N9, ■ ■ ■ — 
this does not lead to any change in what follows because of the stationarity of the field). 

It is not difficult to see that 



i<i<Ne 

= £[Xi.sif]-2a Y Pii/N) + a^£i J2 ^(^') ) + "'^ ( J2 ^' 
i<i<Ne \i<i<Ne J \i<i<Ne 

< l-2a{N9-l)p{9) + a^N^e^ + a^N9p 

= [l - 2aN9p{e) + a^N^9^ + a^N9p] + 2ap{9), (9) 

where we have used the inequality 1 > p(t) > p{9) for r G (0, 9) and the fact that the greatest integer not 
greater than N9 is at least N9 — 1. The value of a that makes the bracketed expression in (|9]l smallest is equal 
to a* = j^g^^p (we do not optimize the entire expression for simplicity). Substitution of this value yields 

p'iO) 2 



l+p/{N9) \ N9 

Now let e > be sufficiently small so that 9'^ — e9{l + 6*) > 0, and let N be sufficiently large so that < e. 
We can always do this since 9 only depends on D and on the autocorrelation function. Now suppose that 
p/N = 9^ - e9{l + 9), then 

1 pIi^(,_A_] < 1 ^ 



l+p/{N9)\^ N9 J ^ ^ l+p/{N9)'''^ '^^ 

1 + 9 - 



The above implies that for N sufficiently large, ■i:pniax(-^, D,p) > 9^ — e6{l + 9). Taking the liminf, we 
obtain that for all sufficiently small e > 0, 

liminf ^Pm..{N, D, p) > 9^ - e0(l + 9). 

Since e > can be arbitrarily small, we obtain the desired conclusion. 

The purpose of this Lemma is only to establish that Pmax(^, D, p) grows at least linearly with N 
constants presented were chosen for simplicity of presentation. 

The following is our main result on the rate of distributed coding: 

Proposition 1 The sum rate of the distributed coding scheme described above is bounded above by a con- 
stant, independent of N . 

Proof Consider a vector Gaussian channel with input W e and output Y G M^, Y = W + Z, where Z 
is as above, and where the power constraint on the input is given by £'[W-^W] < N . Since Z is distributed 
N{0,pl), the capacity of this channel, 

max/(W; W + Z) subject to £:[W'^W] < N, 
w 

is equal to ^ log + (see, for example, ITSl ). 

Let e > be any number smaller than D„et- We know from Section l23] that there is an A^i such that for 
N > Ni, D'{N) > Dnet — £■ Further, from Lemma[T] we know that there exists some N2 > and a constant 
9 > such that for N > N2, PmaxiN, Dnet ^ CjP) > 9^N. Clearly, Pmax{N,D,p) is a non-decreasing 
function of D, and therefore for N > maxjA^i, N2}, Pma,x{N, D'{N), p) > PmaxiN, Dnet — e, p)- It then 
follows that for N > max{Ni, N2}, 

/(X;X + Z)<flog(l + ^). 

Then, using the inequality log(l + x) < x, and using the result of Theorem[T]to substitute X^^i ^fe ^'-'^ 
/(X: X + Z), we see that 

^ 1 

fe=i 

is achievable. o 

The constants in Proposition [T| have been chosen for simplicity. In general, the rates achievable by dis- 
tributed coding are smaller than the bound found in Proposition[T] 



o 

. The 



3 Comparison with a reference scheme 

In this section, we compare the rate of the distributed coding scheme discussed in Section |2] with a reference 
scheme, which for reasons that will become apparent below, we call as centralized coding. 

The scheme consists of one centralized encoder /, which has access to samples taken at all sensors at 
times {1, . . . ,m}, and N decoders, {gk}k=i '^he fusion center The encoder maps the samples of the 
sensors, X(^' - '™)(si, . . . , sat), into an index chosen from the set {1, 2, ... , [e™^"J }, where i?^ is the rate 
of the centralized scheme, and communicates this index to the fusion center The decoder gk at the fusion 
center reconstructs the samples from sensor k from the messages received from the centralized encoder, 

forfc= 1,...,7V. 

At the fusion center, the reconstruction of the field X{s) is obtained in the same two-step manner de- 
scribed in Section lZ2l the fusion center constructs estimates X{sk) of the samples X{sk), for k = 1, . . . , N 
from the messages received from the sensors, and then interpolates between samples using (|3]i. 
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Let R^{Dnet) be the smallest rate for which there exists an encoder / and decoders {gk}k=i such that the 
integrated MSB ([T]i achieved by the above scheme satisfies the constraint (|2]i. Then, it is clear that i?^ (i^net ) 
is a lower bound on the rates of all schemes which use the two-step reconstruction procedure of Section |272l 
In this section we bound the excess rate of the distributed coding scheme of Section[2]over the rate i?^ (-Dnet ) 
of the centralized scheme. 



3.1 Error analysis 

Using the lower bound in Appendix [A) equation ( [22] l. on the error ([T]i in terms of J'f^ig^ ("^) of ® 
conclude that for N large enough, if JMssini) < Dnet, then J'^sEi''^^) — D"{N), where 



_ 2 (1 - (M + V(l - im)) (1 - im) + Dnet) + Due 
^ ^ P' (^) 

Note that D"{N) approaches Dnet from above as iV ^ oo. 



3.2 Bounding the rate loss 

Now, consider 

V*=arg min ^/(X;V), subject to [||X-V||2] < i:>"(7V). (10) 

From Section im it is clear that the rate of the centralized coding scheme, R*j^{Dnet) satisfies, for any N, 

i?^(^„et) >/(X;V*). 

We now use techniques similar to those in fT9l to bound the redundancy of distributed coding over the 
rate of joint coding. Let Z be as in Proposition[T] Expanding /(X; X + Z, V) in two ways, we get /(X; X + 
Z) + /(X; V|X + Z) = /(X; V) + /(X; X + Z|V), so that 

/(X;X + Z) -/(X;V) < /(X;X + Z|V) (11) 

= /((X-V);(X-V) + Z|V). 

Since V ^ (X - V) ^ (X - V) + Z, we have /((X - V) ; (X - V) + Z|V) < /((X - V) ; (X - V) + 
Z). Subject to the constraint in ( fTOb . /((X — V) ; (X — V) + Z) is upper bounded by the capacity of a 
parallel Gaussian channel, with noise Z and input W = X — V, the power constraint on which is given 

by i£[||W||2] < D"{N). The capacity of this channel is HI C = f log (^1 + and therefore 

from (fTTT l and the definition ( fTOb of V as the rate-distortion achieving random vector, we get 

N ( D"(N) 
/(X;X + Z)-i?^(i^„et) < -log'^ ' ^ ' 

< 



2 

N D"{N) 



where the second inequality follows because log(l + x) < x. From Section [TTl we know that for any e > 0, 
there is a Ni large enough so that for all N > Ni, D"{N) < Dnet + and we can choose the variance 
p of the entries of Z to be at least N9^, where 6 is as in Lemma [1] while still ensuring that X + Z meets 
the requirements on the auxiliary random variable U of Theorem [T] Therefore, substituting X^iLi 
/(X; X + Z), and using Lemma[T]and the result of Section lTTI we get that for any e > 0, there is an iVi large 
enough so that for all N > Ni, 

J2R,^R%{Dnet) < (12) 

1=1 

We conclude that the rate of the distributed coding scheme of Section |2] is no more than a constant 
(independent of N) more than the rate of a centraUzed coding scheme with the same reconstruction procedure. 
Again, the constant in ( fT2b has been chosen for simplicity of presentation and is in general much larger than 
the actual excess of the rate of the distributed coding scheme (see Section|5]i. 



4 Point-to-point coding 



The distributed coding scheme studied in Section [2] shows that the tradeoff of sensor numbers to sensor 
accuracy is achievable. However, it may not be feasible to implement complicated distributed coding schemes 
in simple sensors. In this section we show that if the sensors are synchronized and if a delay that increases 
linearly with the number of sensors is tolerable, then the desired tradeoff can be achieved by a simple scheme 
in which encoding can be performed at sensors without any knowledge of the correlation structure of the 
field. 

In this scheme, we partition the interval [0, 1] into A'equal sized sub-intervals, [0, j^], -^],. . .,(-^^^, 1]. 
We specify K later, but assume that N > K sensors are placed uniformly in [0,1]. We assume that K divides 
N for simplicity (so that there are an integer number, ^, of samples in each interval). 

Since the somewhat involved notation may obscure the simple idea behind the scheme, we explain it 
before describing the scheme in detail. We consider time in blocks of duration ^ units each. The scheme 
operates overall with a blocklength of m = "i'"^' th^t is> ™' blocks, for some integer to'. Each sensor is 
active exactly once in any time interval that is ^ units in duration. A sensor samples the field at its location 
only at those times when it is active. Each sensor uses a point-to-point code of blocklength m' and rate Rp 
nats per active time unit. The code is chosen appropriately so as to meet the distortion constraint. However, 
since the sensor is active only in m' out of m'^ time units, the rate of the code per time-step is only ^Rp 
nats. We show below that the desired distortion can be achieved with a rate Rp that is independent of N and 
therefore the desired scaling can be achieved by the above scheme. 

We now describe the scheme in detail. Consider the time instants |1,2,...,to'^}. Each sensor uses 
a code of blocklength to = to'^, which is constructed from a code of blocklength m', as follows. For 
each j in {1, 2, . . . , ^} and each / in {0, 1, . . . ,K — 1}, sensor + j (which is the j-th sensor from the 
left in the sub-interval (-^, and is at location sn;^^) samples the field only at times Ti^ = {j,j + 

+ . . . , j + }. It uses a code of rate Rp, to be specified below, to map the to' samples 

{X'^*^(siV;^^), i e 11. j} to an element of the set {1, 2, . . . , [e™ ^^J }. The rate per-time unit of each sensor 
is therefore — ]-jrm' R„ = ^fR„ nats. 

The fusion center consists of N decoders, one for each sensor Decoder k constructs estimates of the 
samples encoded by sensor k using only messages received from sensor k. Then, for each time i ~ + J 
in {1, . . . , to'^}, the fusion center has reconstructions 

that is, one reconstruction for each sub-interval. 

For any s e [0, 1], we denote the location of the (unique) sensor active within the interval {j^, to 
which s belongs by r^'^ (s). For each time instant i, the fusion center reconstructs the field for s ^ r*^*^ (s) as 

X'^'\s) = pis - rW(s))xW(r(*)(s)), 

where X*^*' (r'*) (s)) is the decoded sample at the fusion center of the sensor at (s) at time i. 
We show in AppendixiBlthat 

< (l-P^(^)) + ^E{i7 E £[{x^'^\s,) - X^^^Hs,))']] (13) 

k=l I jfcGTfc J 

where, with some abuse of notation, we use Tk to denote the set of time steps in which sensor k is active. 
Note that the cardinality of Tk is to' for each k. 

We now choose K large enough so that (1 — p'^ij^)) < Dnet and choose 



Dk ^ D^,t - (l - p\j^)). 
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(14) 




Figure 1: Linear increase of pma^; for large N: p{t) ~ sinc(T) (left) and p(t) = exp{— |t|} (right). Dnet = 
0.1. 



The jTi'-blocklength code used at sensor k for the times that it is active is a code that achieves the rate- 
distortion bound for the distortion constraint 



as m' — > OD. It is well known that the rate of this code is i?p = | log jj^ nats per time step. It is clear 
from (fTSl i and (fl4] i that this scheme achieves the required distortion. Since the rate of each sensor in the 
overall scheme is ^Rp nats per time step we have therefore constructed a scheme in which the bit rate of 
each sensor is 



Kl 

log 

N2 ^ 



(15) 



nats per time step. We can now choose K to minimize the sum-rate — -y log [Dnet — (1 ^ ■ 

Further, it is well known (see 1201 Section 5.1]) that using scalar quantization, each sensor can achieve 

distortion Dk at rate i log — h S, where S is a small constant. For example, for Max-Lloyd quantizers 

(see 11201 Section 5.1]), S is less than 1 bit. 

Therefore, we conclude that it is indeed possible to achieve the desired tradeoff between sensor numbers 

and the per-sensor rate even when the sensors encode their measurements using appropriate scalar quantizers, 

given that we also make use of the synchronization between sensors to activate sensors appropriately. This is 

in contrast to the conclusions of fll, where full use of synchronization is not made, and therefore it is found 

that the above tradeoff is not achievable with scalar quantization. 



5 Numerical examples 

In this section we give numerical examples of the rates of the coding schemes discussed in Section[2] Section[3] 
and Section m The two fields we consider as examples are (1) a (spatially) band-limited Gaussian field, for 
which p(t) = sinc(T), where sinc(T) = ^i^^^, and (2) a Gauss-Markov field, for which p(r) = cxp{ — |t|}. 

For these fields, we numerically find the largest value p^ax of the variance p of Z for which the error for 
the estimator in ^ is no more than the distortion D'{N) of (|5]l, with Dnet =0.1. The resulting values are 
shown in Figure[T] We see that for large values of N, Pmax is indeed approximately linear in N. 

We compute the achievable sum rate of the distributed source coding scheme, which is equal to /(X; X + 
Z) from Theorem[Tl with the Pmax found above as the variance of the entries of Z. These rates are shown 
in Figure |2l For reference, we also show the lower bound on the rate of the centralized coding scheme 
computed in Section[3] 

In comparison, on minimizing the rate (fTsT i of the point-to-point coding scheme of Section H] we find 
that best sum rate for p{t) = sinc(r) is 11.77 nats for K ~ 7 intervals, and that the best sum rate for 
p(r) = cxp(— |r|) is 46.92 nats with K = 24 intervals, which is significantly greater than the sum-rate of the 
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Figure 2: Rates of joint and distributed coding (in nats per snapshot) vs. number of sensors N: p{t) = 
sinc(r) (left) and p{t) = cxp{-|t|} (right). Dnet = 0.1. 



distributed coding scheme found above. However, part of the reason for the large sum-rate of the point-to- 
point coding scheme is that our analysis exaggerates an edge-effect for the sake of simplicity: In Section|4]we 
estimated the value of the field at point s at time i using the sample that the fusion center has at time i from 
the sub-interval that s lies in. We could instead have used the sample closest to s that is available at the fusion 
center at time i, similar to what is done in Section|2]and Section[3] However, this would have meant dealing 
with the first and the last sub-interval differently, and therefore we did not follow the analysis outlined above. 
Without this edge effect, the rates of the point-to-point coding scheme are approximately half the rates found 
above, which are still considerably larger than the sum-rates of the distributed coding scheme. 



6 Conclusions 

We have studied the sum rate of distributed coding for the reconstruction of a random field using a dense 
sensor network. We have shown the existence of a distributed coding scheme which achieves a sum rate 
that is a constant independent of the number of sensors. Such a scheme is interesting because it allows us 
to achieve a per-sensor rate that decreases inversely as the number of sensors, and therefore to achieve small 
per-sensor rates using a large number of sensors. 

In obtaining bounds on the sum rate of distributed coding, we made full use to the heavy correlation 
between samples of the field taken at positions that are close together. When the number of sensors is large, 
the redundancy in their data can be utilized by coding more and more coarsely: this corresponds to more 
noisy samples, and is manifested in the growth of the noise Pmax in the forward channel in Section |2l We 
believe that this technique of bounding the sum rate is of independent interest. 

We have also shown that contrary to what has been suggested in fl] and (31, it is indeed possible to design 
a scheme that achieves a constant sum rate with sensors that are scalar quantizers, even without the use of 
distributed coding. This scheme, however, requires that we make appropriate use of the synchronization 
between the sensors, results in a delay in reconstruction which increases linearly with the number of sensors, 
and achieves rates that may be significantly higher than the rates achieved by distributed coding. The scheme 
is nevertheless interesting because its low complexity makes it easy to implement. 
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A Bounds on Jmse{'>^) for the schemes in Section |2] and Section |3] 

We can write the error in reconstruction at any s E [0, 1] as 

X{s)-X{s) = X{s)- p{s-n{s))X{n{s)) 

= [X{s) - p{s ~ n{s))X{n{s))] + [p{s - n{s)) (x{n{s)) - X{n{s))) 

= Es{s) + Eq{s), (16) 

where i;s(s) = X{s)- p{s-n{s))X{n{s)) and£'Q(s) = p{s~n{s)) [x{n{s)) - ^(^(s))^ Note that in 
the schemes described in Section[2]and Section[3j the encodings of all samples are used to obtain the estimate 
X{n{s)), and therefore X{n{s)) is in general not independent of X{sk), for ^ n{s). As a result, Es{s) 
and Eq{s) are in general not independent. In this appendix, we find upper and lower bounds on JMSsi'm) 
that hold for the schemes of Section[2]and Section[3] 

Using the Cauchy-Schwarz inequality (for any two appropriately integrable random variables A and B, 
\£[AB]\ < ^/£[A^]£[B^), it is easy to see that 



£ {Esis) + EQis)f < £ {Es{s)f + £ {EQ{s)f + 2^ £ {Es{s)f £ {EQ{s)f (17) 
£ {Es{s) + EQ{s)f > £ {EQ{s)f -2^£ {Es{s)f £ {EQ{s)f . (18) 
Now, note that £ {Es{s)f = (1 - ^^(5 _ n{s)). Therefore, 

£ {Es{s)f £ {EQ{s)f = p\s - n{s)) (l - p\s ~ n[s))) £ (x{n{s)) - X[n{s))f . 

For N large enough so that both p^ ( 27v) — 1 ^"'^ ^ / (2^) li^s in the interval around in which p is non- 
increasing (so that for s € [j^, ^) p'^{s - n(.s))(l - p'^{s - n{s)) < p'^{^){l - p^i^)), which holds 
because the function h[x) = a;(l — x) is decreasing in [i, 1]), we get that 

£{Es{s) f£{EQ{s) f < p2(^-L^ (^l-p2|^-L^^f (19) 
From ([T]i and ( fT6] l, we have 

JMSE{m) = - E ^ (4'^ (^) + E^Q (s)) ds. (20) 

Therefore, integrating ( fTTI l and ( fTSl ) over [0, 1], using ( fT9] l and Jensen's inequality (and the concavity of the 
function y{x) = -Jx), and averaging over the time index, we get 



JMSE{m) < |l - (^^) I + J'MSEim) + ^\lpH^)i^ - PH^))JMSEim)^ (21) 
JMSEim) > p^{±-)J',,g^{m) - 2yp2(_L)(i _ p2(_L))4^^^(^), (22) 
where Jmse^'"^) 1^ 1" ®- 

B Error analysis for the point-to-point coding scheme 

With some abuse of notation, we can still write the error in reconstruction as 

X{s)^X{s) = Es{s) + Eq{s), 
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where now 



Es{s) = X{s) ^ p{s - r{s))X{r{s)), ^nd 
Eq{s) = p{s-r{s))[x{r{s))-X{r{s))) . 

In the point-to-point coding scheme, the fusion center estimates the samples of each sensor using only the 
messages that it receives from that particular sensor. Note that Eg^ (s) is the error in the optimal MSE 
estimate of X(s) given X^*)(r(s)). It is well known that if {X(s), s e [0, 1]} is a Gaussian process, the error 
Eg\s) in is independent of X^*' (r^*-* (s)). Further, due to the independence of the field X^*) and the field 
X^^^ for any j ^ i, Eg \s) is independent of X'^^^^A^'> (s)) for all j, and hence also of the reconstructions 
XU) (s)) and the error terms E^q (s). Therefore, for any i, 

£[(xW(s)-l«(,s))^]=£[(4^)(.)f]+f[(£;«(s)f]. 

Now, for /v large enough, £[(£'^'^(s))^] = I — [s — r^^\s)) < 1 — ( ) for every s G [0, 1]. Also, since 
p2(s) < IforallsG [0,1], 

£[iE^^\s)r] < f[(xW(r«(.))-lW(r« (.)))']. 

So, we get 

K-l 



£:[(X«(s)-X«(s))2]ds = J2 L f[(xW(s)-XW(s))2]rfs 

1=0 T< 
A' — 1 i±l 

i=a -k 

1=0 ^ ^ 



where we note that by our notation, r'^^\^-^) is the location of the (unique) sensor active at time step i in the 



interval (-^,^]. 

Now summing over the time index we get. 



-Y, / f[(xW(,s)-xW(s))V 

^ ,=1 -^0 



' K Km ^ ^ \ K K 

1=1 1=0 \ 

Rearranging the sum on the right and substituting m = ^^liY. vve get 



11^ 2 

^ (i-^'(x)) + ^E E £[{x^'^Hs,) - xi'^^Hs.)) ], 

fc=l ifcSTfc 

= (i-^^4)) + ^E(;J7 E £[{x^'^\s,)-xi^^Hs.)y]] 

k=l I ifcGTfc ) 



where Tk is the set of time steps in which sensor k is active. 
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